A Comparison of Linear and Nonlinear Factor Analysis in Examining the Effect of a Calculator Accommodation on Math Performance

Calculator Accommodation 2 2 Abstract In a statewide achievement test, the use of a calculator is often allowed as an accommodation for students with disabilities (SWD). The purpose of the accommodation is to eliminate the disadvantage faced by SWDs relative to peers without disabilities with the same level of overall mathematics proficiency. However, it is important to determine if the accommodation has changed the construct and meaning of the test scores. One way of examining whether the construct has changed due to the accommodation is to compare performance on the assessment for SWDs to peers without disabilities, controlling for differences in proficiency. In this study, measurement invariance of a statewide 8 th-grade mathematics test was explored to evaluate the comparability of the scores obtained by examinees who were accommodated with calculators and those who were not. Structural equation modeling (SEM) was used to assess measurement invariance at the content strand level. Also, nonlinear factor analysis (NFLA) was used to assess the invariance at the item level and the solution was compared to the SEM results. Results indicate the accommodation tended to make items in the content strand Number Sense and Operations easier. The relative strengths and limitations of using SEM and NLFA to evaluate test accommodations is discussed. Standardized tests are designed to promote fairness in testing by keeping the test content, administration conditions, and scoring processes consistent for all examinees. However, for examinees with disabilities, one or more features of the standardized conditions may make it more difficult for the student to demonstrate their true knowledge and skills. For this reason, many testing programs offer accommodations to the standard testing conditions to students with disabilities. The purpose of these accommodations is to remove any barrier to valid assessment of students' knowledge and skills. However, concerns over the degree to which the accommodation may change what is measured by the test, warrant research into the degree to which test accommodations affect the validity of the interpretations derived from test scores. For students who are unable to calculate at any level of difficulty, one common nonstandard accommodation is the provision of a calculator. In order to receive this accommodation, the students must be virtually unable to perform calculation. Without this accommodation, the scores for these students may lead to underestimates of their true mathematics proficiency. In this study, we examine whether a calculator accommodation changes the measurement properties of a large-scale, …